Evaluation of Risk Prediction with Hierarchical Data: Dependency Adjusted Confidence Intervals for the AUC

The area under the true ROC curve (AUC) is routinely used to determine how strongly a given model discriminates between levels of binary outcome. Standard inference with AUC requires that outcomes be independent each other. To overcome this limitation, method was developed for estimation variance in setting two-level hierarchical data using probit-transformed prediction scores generated from generalized estimating equation models, thereby allowing application inferential methods. This manuscript presents an extension approach so may performed three-level (e.g., eyes nested within persons and families). A accounts effect tied on also described. performance 95% confidence intervals around assessed through simulation clustered multiple settings, including ones variable cluster sizes. Across all actual interval coverage varied 0.943 0.958, ratio theoretical empirical 0.920 1.013. results are better than those existing Two examples applying proposed methodology presented.